Blog Tags GitHub

KV Cache

2 posts tagged with "KV Cache"

Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering

A training-free framework for revisit consistency in autoregressive generative rendering using correspondence-guided memory and geometric attention priors"

Generative Rendering Video Generation Revisit Consistency KV Cache Autoregressive Models 3D Scene Understanding Depth Conditional Diffusion

Robust KV Cache Management for LLM Serving under Output Token Length Uncertainty

提出了一种基于Wasserstein DRO的鲁棒KV cache管理框架，联合优化GPU并行配置、KV cache预留、请求路由和前缀缓存，在输出token长度不确定性下实现自适应内存分配和尾延迟控制。核心贡献包括临界分位数结构理论证明、BCD-DRO分解算法和滚动时域自适应策略。

LLM Serving KV Cache Distributionally Robust Optimization Wasserstein DRO GPU Resource Management Queueing Theory LLM Inference